Circumscribing is not excluding: A response to Manning

Manning [1] presents an extended critique of a paper by Zaenen, Karttunen and Crouch [2] (henceforth ZKC), which was a commentary on the PASCAL Recognizing Textual Entailment (RTE) challenge [3]. The ZKC paper was entitled “Local textual inference: can it be defined or circumscribed?” and argued that the PASCAL data should have been better defined and circumscribed so as to distinguish between different forms of textual inference. Some misunderstanding has arisen, and Manning takes us to be in favor of excluding certain forms of textual inference rather than just distinguishing between them. In this note we clarify how the considerations put forward in ZKC serve not to narrow the textual inference task, as Manning claims, but to broaden it. We illustrate this by discussion of the AQUAINT Knowledge-based Evaluation (KBEval) data and annotation guidelines [4], which we played a large role in formulating.